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DETAILED ACTION 



Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set forth 
in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office 
action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed 
on 01/28/2008 has been entered. 



Response to Amendment 

2. The amendments, filed on 01/28/2008, have been entered and made of record. Claims 
1-4, 6-11, 13-15, and 17-32 are pending. 



Response to Arguments 

3. Applicant's arguments with respect to claims 1-4, 6-11, 13-15, and 17-32 have been 
considered but are moot in view of the new ground(s) or rejection. 



Claim Rejections - 35 USC §103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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5. Claims 1 and 3-32 are under 35 U.S.C. § 103(a) as being unpatentable over Gvili, 
"Depth Keying" SPIE Vol. 5006 (2003) in view of Parker, US 7,106,366. 

Regarding claim 1, Gvili discloses a method for differentiating between 
foreground objects and background objects within a scene being captured through an 
image capture device (a novel depth video camera), comprising: 

emitting a ray of light from a light source toward an object of the scene 
(generating a light wall, Para 3.1, pp 566-567); 
opening an aperture cover allowing access to a sensor of the image 
capture device for reflected light from the light source (deploying a 
fast image shutter in front of the CCD chip, page 567, lines 3-4); 
- closing the aperture cover after a set time (the shutter is precisely 
controlling the exposure time of the CCD, page 567, lines 17-18), the 
predefined amount of time corresponding to a maximum distance 
traveled by the light (a real-time trimap is generated for each frame 
based on the original depth matte, page 569, lines 22-23); 
generating a depth mask identifying objects within a foreground region 
and a background region of the scene (foreground objects can be 
generated by setting the depth measurement window, page 568, 
section 3.2) based upon the light captured during the set time (a real- 
time trimap is generated for each frame based on original depth matte, 
page 569, 4th paragraph), the depth mask identifying objects within 
the foreground region with a first bit value (full value pixels 
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representing foreground pixels in the color channel) and identifying 
objects within the background region with a second bit value (zero 
value pixel representing background) (page 564, 6 th paragraph); and 
adjusting image capture device parameters according to bit values of 
the depth mask prior to capturing a subsequent corresponding image of 
the scene (normalized depth of pixel D(i,j) can be calculated, page 
567, lines 7-1 1) wherein the image capture device parameters are 
selected from one of focus (zoomed, page 568 line 2), brightness 
(collected light by each pixel, page 567, line 5), exposure (exposure 
time, page 567, line 18) or gain (normalized depth of pixel, page 567, 
line 8). 

Gvili does not specifically teach that the adjusting of the image capture device 
parameters being done independently in the foreground from the background. 

However Parker does teach the adjusting of the image capture device parameters 
(digital image data outputs from sensor signal processing operation 204 for 
calculation of belief map 209, Fig 3) being done independently in the foreground 
from the background (main subject detection unit 208 has a number of subtasks 
including region segmentation, perceptual grouping, feature extraction, and 
probabilistic reasoning for the subjects in the foreground) (col. 5, lines 14-67 and Figs 
2a, 2b, 2c, and 3). 

Thus, it would have been obvious to one of ordinary skill in the art to have 
included a main subject detection unit as taught by Parker into Gvili 's digital camera, 
as the Parker reference suggested there may be other information relating to the 
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importance of the photographed subject and corresponding background regions that 
can be used by the main subject detection unit to refine the belief map (col. 8, lines 
20-34, Parker). 

Regarding claim 3, Gvili discloses the light source is configured to emit 
infrared light (IR laser diodes, page 567, lines 15-16). 

Regarding claim 4, Gvili discloses a method operation of opening an aperture 
cover allowing access to a sensor of the image capture device (the shutter is 
precisely controlling the exposure time of the CCD, page 567, lines 17-18) 
includes, receiving reflected light from the objects within the foreground region 
(light reflected from every object inside the depth measurement window, page 
568, second paragraph of section 3.2). 

Regarding claim 6, Gvili discloses a method operation of adjusting image 
capture device parameters according to bit values of the depth mask prior to 
capturing a subsequent corresponding image of the scene (ability to change the 
parameters of the depth window according to the scenario's need, page 568, 
section 3.2, first paragraph) includes, determining an optimal amount of light 
based upon the depth mask; and adjusting the aperture cover to allow the optimal 
amount of light into the image capture device (normalized depth of pixel can be 
calculated, page 567, lines 7-1 1). 

Regarding claim 7, Gvili discloses the image capture device parameters are 
adjusted through mechanical adjustments (depth key setting can be automatically 
set, page 568, third paragraph of section 3.2). 
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Regarding claim 8, this claim recites same limitations as claim 3. Thus it is 
analyzed and rejected as previously discussed with respect to claim 3 above. 

Regarding claim 9, Gvili discloses a method for adjusting image capture 
settings for a single image capture device (a novel depth video camera), 
comprising: 

- identifying a scene (easily identified, page 569, third paragraph); 
capturing an image of the scene through the single image capture 
device (the depth information is captured by a camera, page 566, fifth 
paragraph); 

generating a depth mask of the scene from data defining the image of 
the scene (generating depth map according to some depth criterion, 
page 568, last paragraph); and 

- adjusting pixel values of the data defining the image corresponding to 
objects within any one or both of a foreground region and a 
background region of the captured image (normalized depth of pixel 
D(ij) can be calculated, page 567, lines 7-11), and the adjusting of 
pixel values is according to bit values of the depth mask (full value 
pixels representing foreground pixels in the color channel and zero 
value pixel representing background) (page 564, 6 th paragraph). 

Gvili does not specifically teach that adjusting pixel values associated with the 
foreground region are independent of adjusting of pixel values associated with the 
background region. 
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However Parker does teach adjusting pixel values associated with the foreground 
region are independent of adjusting of pixel values associated with the background 
region (main subject detection unit 208 can adjusting pixel values associated with the 
foreground region independently because 208 has a number of subtasks including 
region segmentation, perceptual grouping, feature extraction, and probabilistic 
reasoning for the subjects in the foreground only) (col. 5, lines 14-67 and Figs 2a, 2b, 
2c, and 3). 

Thus, it would have been obvious to one of ordinary skill in the art to have 
included a main subject detection unit as taught by Parker into Gvili's digital camera, 
as the Parker reference suggested there may be other information relating to the 
importance of the photographed subject and corresponding background regions that 
can be used by the main subject detection unit to refine the belief map (col. 8, lines 
20-34, Parker). 

Regarding claim 10, Gvili discloses the method operation of generating a 
depth mask of the scene from data defining the image of the scene includes 
segmenting the foreground and background regions of the scene (automatically 
segmentation of the scene is possible, page 568, third paragraph of section 3.2). 

Regarding claim 11, Gvili discloses the data defining the image of the scene 
includes pixel data where each pixel is tagged with distance information 
(calculate color distances between neighboring pixels, page 570, section d). 

Regarding claim 13, Gvili discloses the image capture device is selected from 
the group consisting of a digital camera, a web cam, and a camcorder (a novel 
depth video camera, page 564, line 4). 
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Regarding claim 14, Gvili discloses the displaying a portion of the image of 
the scene having adjusted pixel values (a new matte is generated by combing the 
information from both the depth and the color frames, page 569, fourth paragraph 
and also page 571, first paragraph of section 4). 

Regarding claim 15, Gvili discloses the portion of the image of the scene is an 
image of a participant for use in an interactive gaming application (man-machine 
interactions, page 572, second paragraph of section 5). 

Regarding claim 17, this claim differs from claim 9 only in that the claim 9 is 
a method claim whereas claim 17 is an apparatus. Thus the apparatus claim 17 is 
analyzed and rejected as previously discussed with respected to claim 9 above. 

Regarding claim 18, Gvili discloses the depth mask is a bit mask having a first 
logical value (full value pixels) assigned to represent the foreground objects and a 
second logical value (zero value pixels) assigned to represent the background 
objects (full value pixels representing foreground pixels in the color channel and 
zero value pixel representing background) (page 564, 6 th paragraph). 

Regarding claims 19, this claim recites same limitations as claim 4. Thus it is 
analyzed and rejected as previously discussed with respect to claim 4 above. 

Regarding claim 20, Gvili discloses each logic element is one or a 
combination of hardware (i.e. shooting lens, depth sensor, as well as a camera) 
and software (i.e. a depth map, depth measurement window, and scene 
segmentation) (pp. 568-569, section 3.2 depth key setting) 

Regarding claim 21, Gvili discloses the image capture device is a video 
capture device (a novel depth video camera, page 564, line 4). 
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Regarding claim 22, Gvili discloses the depth logic is further configured to 
periodically provide a depth mask for a sequence of video frames captured by the 
video capture device (step f of the process for iteration of alpha values, page 570). 

Regarding claim 23, Gvili discloses the image capture device setting is 
adjusted through one of a mechanical or electrical adjustment (depth key setting 
can be automatically set, page 568, third paragraph of section 3.2). 

Regarding claim 24, Gvili discloses the image capture logic is further 
configured to adjust each pixel of image data of the scene (step d of the process 
shown the color distance calculation is performed for each pixel of the image data 
of the scene)( step d of the process for iteration of alpha values, page 570). 

Regarding claim 25, this claim differs from claim 1 only in that the claim 9 is 
a method claim whereas claim 25 is an apparatus. Claim 25 further differs from 
claim 1 in that the limitations "a computing device" and "a display device" are 
additionally recited. Gvili teaches a computing device (a Pentium 3 machine, page 
572, section 5) and a display (a video camera, page 564, line 4). Thus the 
apparatus claim 25 is analyzed and rejected as previously discussed with 
respected to claim 1 above 

Regarding claim 26, Gvili discloses the computing device is a game console 
(game input device, page 572, section 5). 

Regarding claim 27 and 28, these claims recite same limitations as claims 22 
and 23, respectively. Thus they are analyzed and rejected as previously discussed 
with respect to claims 22 and 23 above. 
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Regarding claim 29, Gvili discloses the video capture device is a webcam 
(ZCam is used to generate depth keying during live broadcasts, page 568, line 5 
and page 572, section 5). 

Regarding claims 30 and 32, these claims recite same limitations as claim 11. 
Thus they are analyzed and rejected as previously discussed with respect to claim 
11 above. 

Regarding claim 3 1 , Gvili discloses the scene image data includes an image of 
a person, the image of the person being incorporated into a video game for 
interaction therein (camera using depth-sensing technology are used in man- 
machine interactions) ( second paragraph of Summary, page 572). 
6. Claim 2 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Gvili in view 
of Parker, and further in view of Tuomi, US 7,061,507. 

Regarding claim 2, note the discussion of claim 1 above. Gvili and Parker do not 
teach storing the depth mask in memory of the image capture device. However, 
Tuomi teaches storing the depth mask in memory of the image capture device 
(provided a Z-buffer 1902 for storing the Z-values relating to the depth of the pixel; 
see col. 12, lines, 15-36, Tuomi). 

It would have been obvious to one of ordinary skill in the art at the time this 
invention was made to have used a memory as taught by Tuomi as modified by Gvili 
and Parker so that it provide multiple buffers for storing information (col. 12, lines 
15-17, Tuomi). 



Conclusion 
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7. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: Kopec et al. (US 5,706,364), Berman et al. (US 6,134,346), Long (US 
7,161,634), Okada et al. (US 7,283,679), and Chen (US 6,556,704). 

Inquiries 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kent Wang whose telephone number is 571-270-1703. 
The examiner can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday 
off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ngoc Yen Vu can be reached on 571-272-7320. The fax phone number 
for the organization where this application or proceeding is assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from cither Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see 

http://portal.uspto.gov/external/portal/pair. Should you have questions on access to 
the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217- 
9197 (toll-free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786-9199 (IN 
USA OR CANADA) or 571-272-1000. 

KW 

5 March 2008 
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